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Abstract 

Gateway routers for real-time networks have (he ability to collect delay, loss, and jitter statistics on a pa-connection basis It 
a possible to use this information not only to monitor the quality of ^dividual voice calls and oth C ? real-time Tnn^Ll 
but to evaluate the overall performance of the underlying network. This paper presents describes a method T£ n^,X 
and managing a real-tone data network that supports voice and other real-time services. It is propose? n thatT Rt3 
mechan isms of RTF for sender and receiver reporting be used to relay performance infbmuZ or ml ne^S 
STm"^^ ^ T*?"? 0 * 2) ** a gateWay roWen « and managed XbJZ^^ 

onTariot ScaTes * * ^ *° - d 3 > ™*>™ S 

Introduction 

Network edge devices that perform processing for real-time services can fairly easily collect statistics on delay packet loss 
On^ZlZ^Z*™^* ™ ^ Sd -P-t of dynamic bufS 22^5 

Slf ft SlT^T ^ ^ * ° f m0ait0ring *■ ^ rf «** bein 8 ^vided. On a global J " 
scale toe statistics from all connections can be used to monitor the overall performance of the underlying real-time network, 
providing a picture of "average' network conditions, as well as highlighting trouble spots. It is iZZZuZ Sop 
devtles perfonnaace m ° ml0nn K ■* management system based upon toe per-oonnection statistics collected by the edge 

This disclosure describes such a system. The edge device is a gateway router. In the context of Voice over IP (VoIP) these 
devices are refered to as toe voice gateways, and the data are transported in RTP streams. The RTP specification P ] 
mcludes a control protocol called Real Time Control Protocol, or RTCP. A large part of RTCP is aimed at collect^ 
SSSf ti *! qUflhty 0f ^transport service between session members; i.e. remote applications communicating via RTP 
SSL ^ IT"? ^ KTCP l ° ""^^ *• "l^ant statistics, and defines how thefareVedto 

build toe databases which will support toe network management system. For purposes of illustration, only a voice network is 
considered here. However, the basic design applies to any real-time network; Le, one which carries vidTo sZo^ 
conference calling, multicasting, etc. vmcu, supports 
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High-Level Architecture 

Figure 1 shows a simplified picture of the high-level architecture of the VoIP system and network. AnaW n h™, «.n« „ 
terminated at modems in edge devices called voice gateways. For each call, doming «S^^*J^2f ^ 
coded .and packets by a dedicated modem in one voice m^y,^for^6^ & J^Z^ y Zt^T? 
network to another, remote voice gateway. At the remote gateway me packets are routed from the z^oZtollXY 
moderns, according to an appropriate ID for the phone call. The data in each packet are decoded and rE2L 
the ongmal analog signal (with some associated fidelity), and finally traosmitS to the r«*ij£ iZtfraZ (gr^T 5 
two-way phone calls, this process is obviously symmetric) Omitted from this figure are the JS^to^Z^ 
needed for signalrag, network admission control, etc. <uwu««uraj components 
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Figure 1 



The coded data ate packet,«d m RTP packets, which are themselves transported in UDP packets on the IP network RTP is 
des^nedto opnmue the end-system processing for the real-time nature of the data canied inure RTP nac^tl TL^n? 
RTP packets which is associated with a given phone call is said to belong to an RTP^eiton ^^^00^^ 

^.rf ^ <"* ° f ^ «— * ^ Sphone^S m For 

example, mutople media streams associated with a single video conference would comprise multiple RTP LsbnTwiT 
mutaple parncmants per session. For the purposes of the current document, the scope of an RfiSSto lle 
two-way phone calls.) For agiven session, each participant is classified within me RTP protocol asXr aSSS 
receiver, based upon how recently it has transmitted an RTP packet. sender or 

h^SL £ Z^" ,e K ? hd " 1 " 8 ^ VT « 0C0] > RTCP ' which * h ™ se5si °° members to exchange 
information related to performance, as well as to various signaling functions. RTCP data are earned in RTCP rackets 

These are distinct from RTP packets, but are transported on the same lower-layer protocol (UDP on IP) Each RTP session 
mcludy some proportion of RTCP traffic. That is, RTCP packets associated RTP LioTjP^^l^ 
Of pamcular relevance to the design of a network management system are RTCP sender reports (SRs) and rcretver rerwts 
£Rs). SRs and RRs cany mfcrmatron that can be used to characterize conditions on the IP network caoyS RtT^ 
traffic. Depcndmg upon whether a session member is a sender or a receiver, it periodically transmits RR to all 

other session member, £nly one in the simple VoIP model considered here). Thu S) each 

receives an SRorRR from the other session members). F=n°aicany 

^SRand RR includes one reception block for every source from which the creator of the SR or RR- is receiving RTP 
SSiJfV srmple two-way voice caU, each session member receives from just one source; ie., the other endTf the^ll 

^SSS^J t ! 5 10 reCepUtm h{ ° CKsX a Sender infonnfltion b,ock - ™ e ^ reception block : d£ST" 
statrstical properties of RTP packet reception as observed by the creator of the SRorRR. lite sender mfomationlteck 
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includes data thai describe statical] properties of packet transmission as reported by the creator of the SR. Thus when 
endpomt A recede, an RR from endpomt B, A can detennine how well B "hears" A. For example, A can compare] to 
received packet count wuh the number of packets expected (as inferred from RTP sequence number) Z SELe the 

^ h 5?? ■ Whe T dp ° int A reCCiveS M SR *»» B ' A &«* thesa ™ Nation as kan^nd m 

addmon gets mfonnajon on how many RTP packets B has sent to A. These data can be used to ora 5 B 
Note that a reception block contains most of the information relevant to the quality of the connection P 

The relation between the high-level picture shown in Figure 1 and RTP sessions and streams is shown in Figure 2 Each oair 
of tees between two gateways represents an RTP session terminated at the gateways. As illustrated in this figuU" aonT 
gateway may simultaneously terminate sessions with several other gateways. It is expected that the network oondWoT wOL 
on average be foe same for aU sessions between a given pair of gateways. This sugge-Tfoat copies of 

S° I? K SR ^ dRRS * 3 gatBWay f0r ""»-**>" be maintained at Ac gateway .Further, ^erec^n 

blocks should be ^partitioned according to source gat^ays to which mey apply, and te^^ 
prov.de a charactenzauon or (he network conditions between that source-destination gateway pair. The next secTon 
discusses two alternative approaches to retention of reception blocks by the gateway mat generates them. 




Figure 2 



In order to prov.de a network-wide view of conditions, the monitoring system win include a number of processing and 
momtormg sites for collecting the data on all three time scales. These sites will be organized in a topological hierarchy that 
is defined according to groups of gateways. The hierarchy uses a naming convention of cluster, to refer* groupings of 
gateways, and levels to define placement within the hierarchy. At the lowest level of the hierarchy wc definf a unitcallcd a 
LevelJ duster com^d of z set of gateways, ctUzd Level J cluster members. Each gateway in a Level 0 cluster may talk 
» any ofoer gateway m the Level_0 cluster, so that the LevelJ) cluster also defines every^oTsmle gateway" pair be 
formed by its members^The torn LevelJ cluster pair is used to define a gateway parr for two gateways belong™, to the 
S7nil?-> Cn^ase of multiple, co-located gateways is considered a single, compound gateway in this model) 

The r monitor for g.ven LevelJ) cluster is responsible for monitoring the network conditions between all Level 0 cluster pairs 
m .ts Level_0 cluster. The term LevelJ cluster monitor is used to define this monitor. " 

1CVCl ^""^^V defines a unit called a c/uster. Each Level 1 cluster consists of a specific set 

of LevelJ) clusters, called LevelJ cluster members. We define zLevelJ cluster pair aTapair of gateways forrneTby any 
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two gateways belonging to different Level 0 clusters within the Level 1 cluster tw« M u „ 
pair belongs to a different Level 0 clusterfbut each bu^^mT^^ I ^ * 3 LeVC,J c,uster 
I-eveU cluster has a distinct monitor for aU lW 1 cS LKSJ-? ^ * asaAa ° f ±c Leve1 - 1 A 
*«W is used to define this monitor ~ ^P^^^Level.l duster. The term Levelj duster 

The next higher level of the hierarchy defines a unh called a Lewi iri,.*t~ c*. , -. , 

ofLeveM clusters, called W 2 cluster mSers We defu^B ill ^ ^ achLcveI - 2 ^consists of a specific set 
two gateways be.onging to different Level UhSTers Z^^ltoTfZZ ° ^» * any 
pair belongs to a different Level 1 clmtcrfbiitcach^a^nS That. ^ each gateway m a Level_2 cluster 

Level.2 cluster has a distinct monitor for ^lVvT fS iSSSjlSl ^ 8 ""if ° f Levcl - 2 «*«*. A 
monitor is used to define this moS " ^ pa,rS w,tf,m the Leve1 - 2 duster. The term Lcvel_2 duster 

J-gJL hierarchy. 
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commons being monitored, m particular, problems can be traced to the smauS rde^t^tf 4 *MtehL» t. 

sSlon W tkh L IS f°™ aU °f T 1 be V* 5 * U P * e Wcrareh y * ^me point Related to this is the time 
scale on which data are collected and analyzed. Data collection time scale and processing are described in the next se Son 
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Data Collection and Processing 
Standard RR and SR Features 



The interval between successive RRs and/or SRs from a given session member « a «_ • . 

anupper Ibound of 5% on the fraction of the session ban^^^^^c ^T^S ^ V"** 
adjustment algorithm attempts to scale the interval linearlv with iw „,, m hi „r„«T v m PJ > * e 

session bandwidth dedicated*, RTCP tnffleiitaiSta M^27^T™t SUCh * e ^ m of 

Vpacket floodingwhichcanbepr^^^ 

conference call.) The actual interval used is the mLnum ofTe^^er^S I ^F^'nT' **■ 
call, a very rough estimate of RTCP packet transmission interval can bVobtakS Slows ^tne iZT ^ 

™*- S *™y, t „ccn S ^ 

gateways, assuming the RTCP transmission times for all Ihe sessions are umo^w! 

The monitoring system will provide a view of network conditions and performance cn three time scales- nal «m- fnr , 
conditions; near real-time, current conditions; and daily, for long-term tend an^is A^^!h^T,'J^ ' J 8 ™ 
monitor as soon as possible after they are discovereAFrom the S^aS! ^StZ'h^ 
scale on me order of about 10 seconds, assuming the problem ^^^d^^^Z^T^^Z^T 0 
shorter if the same problem is encountered by several concurrent calls ^^o^I^TtT^ , , madc 
time monitoring should be set such that the fastest, WpaSgkal" SaTb^^^Z h^T ™ ™* 
trend analysis basenupondailym^rmgd^ 

as me system that rcedves me reception blc^^ 

information. However, there are certain quantities that can only be computed bv the svsrem r« - A , 

applies. Le., when system B gets the reception block from STRESS ^ spSnX2^*TT r> 
can use. An example is round trip delay. In mis case, system^ pute a time starn^ ^^n?sL Sa^L^S. 
tnnc between reception of B's SR and A's transmission of its next reception Wock ZS to R 2T£ * "f^ * ,ta 
timeincludedin A's reception block to B. When B gets meXn^ 

its own reception time of the SR or RR that contained the block in order toH^C* * , ^ included **** ^ 
^.thnes^panddelmt^ 

quantmes, such as packet loss and jitter observed at A are useful to any system with accLTft™ln wfi T 
following descriptions of reception block processing, it is assumed thattSte^ ? 6 , 
ccrmputesround trip tune, andthatto 



RR and SR Processing Specific to this Disclosure 

A copy of every reception block generated at a given gateway for inclusion in an <?* ™ pd ct.« u u 
gateway for its monitoring functions. This can be acc^Lhed uWZ^ loJ fl™« f \** 
reception block, a copy could be "diverted" to a monitoring proc s S IStiv^v fan «K « 8 mT^ 5 1,16 
fitted, wlmmeongmatinggateway being des^ 
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require multicasting. Also, this approach could allow alaTSl to be ? ? f 8 ^ ^ il does not 

block, generation, even before the full SR or RR is cJ^SS! * S^fS P . , ° f 
collection of reception blocks within the context of^RTCT ftM ^Sf ? ""J* 1 " ** ft standa ^ the 

htteroperabOitybcco.nesanis.e. Ita^^^S-J^ ™s couidbe intponantif 

wTst^r» 

This tests delay, packet lo£ and jitter ^iSSSSSSf S2S tS2£T aVaflable to Ph ** 0 Passing, 
external alarm processing routine to be Ked^iTi^i^T^ M CauSC 1,16 piocess 10 ^ and an 
-odated gateway pair.Wexe^^ 

statistics for the source gateway to which the reckon block ^3? U S^SKft. 'J"' 8 ^ 

processing. The specific steps included in the each phase of the WeaZ ^Hh!! 1^ ^ **" * c i°*g-tam 

study. Note that the gateway on which this process 5E« £3^!EX £ ta 

even though an SR or RR is intended for traZissio? STJ^Lh^ ttStSlS J? " * 

network conditions as observed at the originating gateway. 21?^^™]'? 

block and reports results to the appropriate monitor. «"S"»nng gateway keeps a copy of the reception 
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R^t^^tt^S?^ shown in Figure 5. In this ^ fc ^ 
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doou^en, B Lb m JSZ^^^ZZZ^J" Kaa °"- " UrdW1 ' — «*• " I. ft. MP 



transmission. 



J originating 
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Figure 5 



^.Tpf^ 0 " ^ m ° re d ^ about the Level 0. 1 , 



and 2 processing. 
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Jitter 

Packet loss: fraction and cumulative 
Receive buffer length 

i? I lL C ? hr ( !f i""*^ is implemented) 
)f the first murnftkM. j ■ ... ... 7 



If any of the first four of these exceed a swcilSdftJS^fS 

ismvoked. TTus routine i, a separate P^^L^T^Z^ *°* * " ^ *»■ routine 
be-spectfiedalgorithn, will be used to avoid flooding S SSJiSZl^^i*^^*^ 1 ^ ^yet-to- 
routme completes, execution of the RTCP receive process^re^ * ,th ** messa S«- Once the alarm 

The first three items are obtained from me reception bloclc wh,v», ™»„ » 
Phase 1 Processing 

Phase ilprocessing uses the Phase 0 data to maintain and update a Phase 1 data W tk- a . 
according to gateway pair. That is, according to the soinrTSLav Til ^ \ ?*? *" base 0r 8 ani » 3 data 
gateway and (he gateway on which the data ^ h ? T Ptl ° n * 0Ct ^ COmbinatio « of the source 

(cluster, supercluster, etc.). The updating V^fSS^S^SZ^ *>* * US monitor 
pair. For the first four quantities, this is just an accumulation^^ * P™ 10 " 5 for this gateway 

applicable) is to be determined. accumulation process. The approbate way to record the current decoder (if 

Periodically (period to be determined), the data associated with earh cutout. 

with the pair. A suggested period is three rnin^s Tt^^^^ * tr ^ smtaed to the monitor associated 
first four items such that at the end of each P^^^llt^ rZ^ ^l * 
transmtssmu, the statistics are reset in preparation ^JL^SS^^^^ 9 ^ 1 ^ ^ 

Phas 2 Processing 

Phase 2 processing uses the Phase 0 data to maintain and update a Phase 7 A** w*„ tv- . . 
the raw data from each session over a Jong interval The Phase2 ^L^T ■ ^ * *" h m ^^on of 
of Phase 1 processing. The interval of the Phase 2 dateU Zk mOOUX? Z ^ ^ ^ P"** 

Phase 2 period, the data base is transferred tome hfe£ 5^325 X? ° D " ° f 0ne At Ae «» of 
long-term trends on a system-wide basis. Transfer ZftJrZ^^Zl ™™ At this site, the monitor can track 

time on the network. 1 y) Phase 2 data may he by FTP during some relatively quiet 
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RT f T-r? 68 f °" 0Win8 HT* delay, jitter, and packet loss- 

• RTT: this is computed using 32-bii time values ^ 

• Jincr: 32 bits 

, Pacl f elloss i : 8 btoforf ^tional loss; 24 bits for cumulative 
auuer length can be specified in 24 bits for less"* i„«u„j <• , . 

and coder is then 32 bits. The total length data b 16 by^ " ^ e combination of bufier length 

Additional information includes the IP address nf**r>h a 

source of any transmission roam niter, hs IP 7 **««■■"•*■ in thepah- is the 

addttumal 32-bit value is needed to include the IP aa^^g^^eX^f ^ °^ one 
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The above infonnation can be represented in a 20-byte block. 
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